Signals and Noise in Drug Safety 
Analyses 

The incretin therapy debate provides the rationale for revamping 
epidemiologic pharmacovigilance 



■ n this issue of Diabetes Care, our edito- 
I rial team has decided to "push the en- 
I velope" by openly addressing a safety 
issue in drug development and monitor- 
ing that is gaining interest and provoking 
controversy. Traditionally, we have not 
felt it was our charge to comment on 
moral issues or political decisions related 
to aspects of medical care or delivery. 
Rather, it has been our goal and privilege 
to "survey the current landscape of clini- 
cal, research, and health care changes" 
and "to provide the most up-to-date in- 
formation for our readers" (1). On one 
previous occasion we felt it was necessary 
to voice our opinion as an editorial on 
academic freedom (2), which we consid- 
ered critical for the sake of scientific di- 
alogue and collaboration across national 
borders. At present, we feel it is our 
charge to bring to the forefront another 
contentious topic that directly concerns 
the care of people with diabetes: the on- 
going debate on the risks versus benefits of 
incretin-based therapies. Thus, in this is- 
sue we present four articles providing 
background material and specific argu- 
ments on this controversy. 

As is currently well known and ac- 
cepted, many parties have a stake in drug 
development, its regulation, and the sub- 
sequent use and monitoring of approved 
agents. Manufacturers of drugs and devi- 
ces have obvious financial incentives to 
develop and market their products. We 
charge the regulatory agencies with guid- 
ing the process, approving safe agents for 
clinical use, and assuring long-term sur- 
veillance of their effects. The charge of the 
medical and scientific communities is to 
assist the pharmaceutical industry in de- 
veloping new therapies, testing them, and 
advising how they can best be used in 
clinical practice. However, as outlined 
in a commentary by Dr. Eric P. Brass in 
this issue of Diabetes Care (3), the input 
of advisory committees is not always 
clear-cut in the approval regulatory pro- 
cess. The charge of regulatory agencies 



includes monitoring the long-term safety 
of drugs that have attained approval for 
clinical use, especially those for which 
specific adverse events have been sus- 
pected. When a drug is released, clini- 
cians must judge how best to use it and 
balance the safety profile, among other 
available therapies, with the aim of pro- 
viding the best "personalized" treatment 
for individual patients. Unfortunately, 
one can argue that the current systems 
to register efficacy and systematically re- 
port adverse events are largely inefficient 
or incomplete. 

Recently, another force has emerged 
in this process: the media. The intensity of 
interest in drug choices has been in- 
creased by direct-to-consumer advertis- 
ing, but uncertainty about possible 
adverse effects from some agents is now 
at the forefront of any discussion. In some 
cases, well-publicized allegations of risk 
from specific agents have provoked wide- 
spread anxiety and distress. Regulatory 
agencies are under immense pressure to ap- 
prove or not to approve agents. Some of the 
ensuing debates have been acrimonious. 
Based on these realities, our editorial 
team has concluded that leading medical 
journals such as Diabetes Care have a re- 
sponsibility to provide leadership while 
supporting clear thinking, good science, 
and civility in the ongoing dialogue. Our 
goal is to provide our readers with the 
most up-to-date information to assist 
them not only in managing their patients 
but also in understanding the public con- 
versation on incretin-based therapies. We 
believe that the weaknesses of the current 
pharmacovigilance system call for re- 
newed attention and improvement. For 
example, prospectively collected data for 
longitudinal epidemiologic analysis and 
well-performed patient-level meta-analyses 
of large, randomized, long-term medical 
outcome trials would do much to support 
evidence-based decisions. Toward this 
end, we hope to stimulate new ideas and 
new methods of addressing important 



questions concerning therapies now in de- 
velopment. 

In this issue, we begin our discussion 
on drug safety with an invited review 
from Prof. CliffordJ. Bailey (4). Targe ran- 
domized interventional trials, population- 
based epidemiology, and medical statistics 
are relatively young sciences that have 
been made possible by the recently ac- 
quired power to store and analyze huge 
collections of data using computers. We 
are still learning how best to use them. 
Methodologic disputes about ways to 
measure medical outcomes, identify 
trends, and assess statistical relevance are 
common and lead to difficulties for clini- 
cal researchers, regulatory agencies, prac- 
ticing physicians, and patients alike. Thus, 
it is important to understand the complex- 
ities of assessing new or current products 
in regard to both benefits and risks. The 
review by Bailey elegantly describes recent 
signals that have been associated with 
diabetes therapies, and his narrative illus- 
trates the difficulties when ascribing cau- 
sality and when evaluating absolute risk, 
predictability, prevention, and contain- 
ment. His article emphasizes that individual 
clinical trials are necessarily restricted 
for patient selection, numbers, and dura- 
tion and that they can introduce allocation 
and ascertainment bias. Further, clinical 
trials often rely on biomarkers to estimate 
long-term clinical outcomes that may not 
necessarily correlate with hard end points. 
As discussed by Bailey, reports of small 
numbers of fatalities or other serious ill- 
nesses associated with specific drugs have 
led to high levels of alarm and, as in the 
case of rosiglitazone, sometimes the with- 
drawal of agents from use. Determining 
whether such associations are the results 
of chance, confounding factors, or the 
drug itself is a statistical problem. Beyond 
that, deciding how to respond to such in- 
formation is a policy issue. All drugs (and 
foods and beverages) have some capacity 
for harm. Consuming too much alcohol 
or too many pizzas over a chronic period 
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obviously can result in adverse effects on 
human health, and even drinking too 
much water can cause illness. Penicillin 
can cause fatal allergic reactions, and aspi- 
rin can lead to death from gastrointes- 
tinal bleeding. Both agents might have 
difficulty passing regulatory review in a 
zero-tolerance regulatory atmosphere. 
The resolution of this problem lies in re- 
garding risk always in the context of ben- 
efit. Of course, the benefits of treatments 
are no less challenging to quantify than the 
harms. 

Even when we have relevant data 
from clinical trials outlining the signals 
and benefits, how to act upon this in- 
formation is not always clear. To illustrate 
this point, the aforementioned commen- 
tary in this issue by Brass (3) addresses the 
approval process for new therapeutic 
drugs in the U.S. He describes the charge 
of advisory committees such as the 
Endocrinologic and Metabolic Drugs Ad- 
visory Committee (EMDAC), which "re- 
views and evaluates data concerning the 
safety and effectiveness of marketed and 
investigational human drug products for 
use in the treatment of endocrine and 
metabolic disorders, and makes appropri- 
ate recommendations to the Commissioner 
of Food and Drugs" (3). Unfortunately, 
there is concern that diabetes clinical trial 
experts may be excluded from these 
committees because of real or perceived 
conflicts of interest related to previous par- 
ticipation in the design and/or conduct of 
any industry-sponsored study. As such, 
one can argue that we are not taking ad- 
vantage of additional valuable expertise 
specific to the drugs being assessed. Brass 
notes that the U.S. Food and Drug Admin- 
istration's (FDA's) Division of Metabolism 
and Endocrinology Products (DMEP) is re- 
sponsible for the review of most drugs used 
for indications relevant to the endocrinolo- 
gist and coordinates the meetings of the 
EMDAC. In his commentary, Brass de- 
scribes how the assessment of these drugs 
requires integration of preclinical and 
clinical data to formulate a benefit-risk 
assessment. Clearly, the assessment of the 
benefit-risk for drugs is increasingly recog- 
nized to be a challenging task, and formal 
tools have been proposed to make this pro- 
cess more effective. It is clear that the FDA 
makes safety the highest priority, and we 
fully endorse that approach. Thus, it is of 
interest to compare recommendations 
made by the EMDAC and those of the 
DMEP. We doubt that many readers are 
aware of the level of concordance between 
EMDAC votes and the subsequent actions 



of the DMEP and the ultimate fate of the 
drug. Brass's comments on this point are 
enlightening, to say the least. 

Finally, Bailey's review and Brass's 
commentary set the stage for the specific 
case of incretin therapies. The Point- 
Counterpoint debate in this issue (5,6) 
bears the overarching title, "A Critical 
Analysis of the Clinical Use of Incretin- 
Based Therapies." It was not a hard de- 
cision to focus on this topic. On the one 
hand, incretin-based medications have 
proven to be effective glucose-lowering 
agents. The glucagon-like peptide 1 
(GLP-1) receptor agonists demonstrate, 
depending on study populations, an effi- 
cacy comparable to insulin treatment, and 
they appear to do so with minimal hypo- 
glycemia and often favorable effects on 
body weight. In addition, there are data 
showing that dipeptidyl peptidase 4 
(DPP-4) inhibitors also improve glycemic 
control with weight-neutral effects and 
with less risk of hypoglycemia than is 
caused by sulfonylureas. Thus, these 
agents appear to address unmet clinical 
needs. However, as highlighted above, 
important concerns have been raised 
about the long-term safety of these agents. 
In addition to preclinical data, some ran- 
domized controlled trials of incretin- 
related therapies have reported small 
imbalances on the frequency of pancre- 
atitis as have isolated case reports after 
recent exposure to incretin-related ther- 
apies. The concern grew exponentially 
with the reports from the MedWatch sur- 
veillance system (the FDA Safety Infor- 
mation and Adverse Event Reporting 
Program), and the controversy expanded 
further with the conflicting reports extracted 
from insurance claims and prescription 
databases not designed to prospectively 
capture adverse events. Specifically, 
based on such reports, it has been sug- 
gested that both of these classes of agents 
have the potential to promote acute pan- 
creatitis, to initiate histological changes 
suggesting chronic pancreatitis including 
associated preneoplastic lesions, and po- 
tentially, in the long run, pancreatic can- 
cer. Other issues relate to a potentially 
increased risk of thyroid cancer. We rec- 
ognize that the data are complex and per- 
haps conflicting, so in keeping with our 
aim of providing a balanced view, we 
present a two-part Point-Counterpoint 
discussion. Prof. Edwin A.M. Gale and 
colleagues describe findings supporting 
their opinion that we should reconsider 
the use of incretin-based therapies because 
of growing concerns of unacceptable risks 



(5). In the narrative following this contri- 
bution, Dr. Michael A. Nauck provides a 
defense of incretin-based therapies, sug- 
gesting that the benefits clearly outweigh 
the concerns of risk (6). Both narratives 
are well written, make excellent points, 
are well referenced, and insightful. 

So, where do we stand after this 
exercise? We believe the excellent articles 
in this issue of Diabetes Care support the 
notion that the drug development process 
is complex, involves many parties, and 
should be guided by science and profes- 
sional discussions rather than inflammatory 
exchanges in the public media. Further- 
more, when available information is lim- 
ited, many different opinions on these risks 
and benefits are possible, but caution 
should be exercised to avoid premature or 
unwarranted positions. A commitment to 
the long-term evaluation of all treatments, 
extending well beyond initial regulatory 
approval, is needed. An important long- 
term goal should be the identification of 
individuals or groups of individuals particu- 
larly suited for — or at risk from — treatment 
with widely used therapies. 

We do not speak on behalf of the 
American Diabetes Association, which has 
allowed us to provide a balanced forum. 
Nevertheless, whether or not incretin- 
related therapies increase the risk for pan- 
creatitis and pancreatic cancer, we believe 
there are pertinent questions that need 
to be properly addressed. Several ways to 
improve pharmacovigilance are proposed 
below. 

1) Given that the current ongoing ran- 
domized cardiovascular (CV) outcome 
trials with GLP-1 receptor agonists (n = 5) 
and DPP-4 inhibitors (n = 4) already have 
a pancreatitis adjudication process, these 
studies would benefit from a separate 
process for identifying and adjudicating 
cancer events. The sponsors of the CV 
outcome trials are encouraged to agree to 
pool patient-level data from the tens of 
thousands of participants in these trials for 
meta-analysis by independent experts. 

2) Creation of a blue ribbon indepen- 
dent expert committee including lead- 
ing cancer epidemiologists, biostatisticians, 
oncologists, gastroenterologists, cardiolo- 
gists, and diabetologists is strongly encour- 
aged. If provided access to all patient-level 
data from the long-term CV outcome trials, 
the group could thoroughly report the 
incidence of pancreatitis, pancreatic 
cancer, thyroid cancer, or any other rel- 
evant adverse outcomes. The most relevant 
professional organizations could take the 
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lead in convening and supporting such a 
committee. 

3) An alliance between all incretin-related 
manufacturers to sponsor the implemen- 
tation and conduct of large, long-term, 
prospective epidemiological studies spe- 
cifically designed to assess the occurrence 
of pancreatitis and pancreatic cancer would 
be very helpful. Countries with reliable 
registries for cancer and diabetes morbid- 
ities as well as electronic medical systems 
where reliable demographic and morbid 
data are available, including pharmacy 
data, will be best equipped to conduct 
such large-scale prospective epidemiologic 
studies. 

4) Finally, we suggest the current pharma- 
covigilance system based on the spontane- 
ous reporting (MedWatch) be re-evaluated, 
with the goal of improving the quality of 
the data collected and the means of adju- 
dicating and better processing the cap- 
tured information. 

In closing, we will restate our belief that 
in reaching conclusions and establishing 
policies regarding the assessment of drug 
safety, all the parties involved (i.e., manu- 
facturers of drugs, regulatory agencies, 
medical and scientific communities, and 
patients) have much to gain from a trans- 
parent and complete evaluation process. 
We hope the articles assembled in this issue 



of Diabetes Care will help all readers to 
judge for themselves the evidence pre- 
sented regarding the incretin-based thera- 
pies and to appreciate the challenges of the 
drug development and monitoring pro- 
cess. We welcome comments on whether 
we have achieved our objectives. 
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